Online optimal and adaptive integral tracking control for varying discrete‐time systems using reinforcement learning

Conventional closed‐form solution to the optimal control problem using optimal control theory is only available under the assumption that there are known system dynamics/models described as differential equations. Without such models, reinforcement learning (RL) as a candidate technique has been successfully applied to iteratively solve the optimal control problem for unknown or varying systems. For the optimal tracking control problem, existing RL techniques in the literature assume either the use of a predetermined feedforward input for the tracking control, restrictive assumptions on the reference model dynamics, or discounted tracking costs. Furthermore, by using discounted tracking costs, zero steady‐state error cannot be guaranteed by the existing RL methods. This article therefore presents an optimal online RL tracking control framework for discrete‐time (DT) systems, which does not impose any restrictive assumptions of the existing methods and equally guarantees zero steady‐state tracking error. This is achieved by augmenting the original system dynamics with the integral of the error between the reference inputs and the tracked outputs for use in the online RL framework. It is further shown that the resulting value function for the DT linear quadratic tracker using the augmented formulation with integral control is also quadratic. This enables the development of Bellman equations, which use only the system measurements to solve the corresponding DT algebraic Riccati equation and obtain the optimal tracking control inputs online. Two RL strategies are thereafter proposed based on both the value function approximation and the Q‐learning along with bounds on excitation for the convergence of the parameter estimates. Simulation case studies show the effectiveness of the proposed approach

Simultaneous Output-Feedback Stabilization For Continuous Systems

A design technique for the stabilization of M linear systems by one constant outputfeedback controller is developed. The design equations are functions of the state and the control weighting matrices. An example of the stabilization of an aircraft at different operating points is given

Optimal Output Tracker Using A Time-Weighted Performance Index

In this paper we develop necessary conditions for the optimality of discrete system feedback controllers using a performance index imposed of (1) time weighting of the states, (2) steady-state error weighting and (3) weighting of the control gain. The controller performance is demonstrated using an F-14 digital pitch rate controller. (C) 1998 John Wiley Sons, Ltd

Optimal Output Tracker Using A Time-Weighted Performance Index

In this paper we develop necessary conditions for the optimality of discrete system feedback controllers using a performance index imposed of (1) time weighting of the states, (2) steady-state error weighting and (3) weighting of the control gain. The controller performance is demonstrated using an F-14 digital pitch rate controller. (C) 1998 John Wiley Sons, Ltd

Simultaneous Output-Feedback Stabilization For Continuous Systems

A design technique for the stabilization of M linear systems by one constant outputfeedback controller is developed. The design equations are functions of the state and the control weighting matrices. An example of the stabilization of an aircraft at different operating points is given

Simultaneous Output-Feedback Stabilization For Continuous Systems

A design technique for the stabilization of M linear systems by one constant output-feedback controller is developed. The design equations are functions of the state and the control weighting matrices. An example of the stabilization of an aircraft at different operating points is given